Risk adjustment: towards achieving meaningful comparison of health outcomes in the real world.

Health outcomes evaluation seeks to compare a new treatment or novel programme with the current standard of care, or to identify variation of outcomes across different healthcare providers. In the real world, it is not always possible to conduct randomised controlled trials to address the issue of comparator groups being different with respect to baseline risk factors for the outcomes. Therefore, risk adjustment is required to address patient factors that may lead to biases in estimates of treatment effects. It is essential when conducting outcomes evaluation of more than trivial significance. Risk adjustment begins by asking 4 questions: what outcome, what time frame, what population, and what purpose. Next, design issues are considered. This involves choosing the data source, planning data collection, defining the sample required, and selecting the variables carefully. Finally, analytical issues are considered. Regression modelling is central to every analytic strategy. Other methods that may augment regression include restriction, stratification, propensity scores, instrumental variables, and difference-in-differences. The construction of risk adjustment models is an iterative process requiring both art and science. Derived models should be validated. Limitations of risk adjustment include reliance on data availability and quality, imperfect method, ineffectiveness when comparators are very different, and sensitivity to different methods used. Thoughtful application of risk adjustment can improve the validity of comparisons between different treatments, programmes and providers. The extent of risk adjustment should be guided by its purpose. Finally, its methodology should be made explicit, so that informed readers can judge the robustness of results obtained.